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ABSTRACT 

Online discussions have been increasingly integrated into face-to-face classes at universities to enhance 
student learning of course content. The primary focus of past research has been on the end products of 
online discussions. Studies reported either successful findings or results that fell short of desired learning 
outcomes. An in-depth investigation is needed about how the design of online discussions is related to 
success. This paper reports our experiences in conducting a three-phase, design-based study: (1) 
designing the online discussion activity; (2) implementing our design in a university course; (3) 
empirically investigating our design. 
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I. INTRODUCTION 

The use of online discussions is gaining momentum in university classrooms. It is widely recognized that 
online discussions have the potential to cultivate and develop student higher order thinking skills [1, 2, 3, 
4, 5]. However, due to inappropriate designs, many online discussions failed to achieve desired learning 
outcomes [6]. “OL might have great potential to promote critical thinking, however, to implement OL, 
and especially online discussions in a way that actualize this potential has proved to be a real challenge” 
[7, p. 808], 

Instructional design plays an essential role in determining successes of online discussions [8, 9]. Higher 
order learning through online discussions requires more than “undirected, unreflective, random exchanges 
and dumps of opinions” [10, p. 14]. It requires the systematic approach in designing a meaningful context 
wherein students practice higher order thinking skills purposely. 


Aviv’s study [1] demonstrated that appropriate design was imperative in promoting knowledge 
construction in online discussions. With careful design and structure, students took an active role in 
constructing knowledge. Student online discussions consequently reached a high level of cognitive 
engagement. Without proper design and structure, students took on a passive role in the discussion; their 
cognitive engagement was low. 
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The primary focus of past research has been on end products of online discussions. Studies either reported 
successful findings or described results that fell short of desired learning outcomes. An in-depth 
investigation is needed about how the design of online discussions is related to success or failure. 


In this paper, we report our experiences in conducting design-based research that contains three essential 
components: (1) the design of the online discussion activity, (2) the implementation of our design in a 
university course, and (3) the empirical investigation of our design. 


Design-based research fits well with our purpose of improving teaching and learning with online 
technology. Education is by nature a designed intervention [12]. Wang and Hannafm [13] defined the 
design-based research as “a systematic but flexible methodology aimed to improve educational practices 
through iterative analysis, design, development, and implementation, based on collaboration among 
researchers and practitioners in real-world settings, and leading to contextually-sensitive design principles 
and theories” [12, p. 6, 7]. Design-based research can trace back to the work of Brown [14] and Collins 
[15]. Design-based research as an approach was conceived out of their concerns that laboratory studies of 
instruction interventions are limited in addressing the dynamics, complexity, and messiness of learning 
processes in real classroom settings. In recent years, design-based research has gained popularity as a 
form of educational research for those who attempt to improve student learning in real classroom settings 
through designing educational interventions, especially involving the use of technology [15, 16, 17, 18]. 
Design-based research has five basic characteristics [13]: 

1. Design-based research is pragmatic. Researchers investigate practical issues to seek 
understanding about design, theory, research and practice. Design-based research considers that 
good theory informs and improves practices. 

2. Design-based research is grounded. In design-based research, researchers ground design in theory 
and implement the design in real-classroom contexts. 

3. Design-based research is interactive, iterative, and flexible. Design-based researchers form 
interactive collaboration with practitioners. The processes involve an iterative cycle of design, 
implementation, research, and redesign. Design processes are flexible to allow emerging insights 
to be incorporated. 

4. Design-based research is integrative. Design-based research integrates a mix of research methods 
both qualitative and quantitative to increase its objectivity, validity, and applicability. 

5. Design-based research is contextual. Results of the design-based research need to be connected 
with both the design process and contextual factors of the setting where the research is conducted. 
Researchers need to document the design process, research findings and changes from the initial 
plan in detail so that other designers and researchers can examine research findings and trace 
contextual factors and conditions that led to the findings. 


We believe that online discussions have the potential of promoting student higher order learning if 
designed appropriately. We decided to experiment with online technology in our course and believe that 
the empirical study of our design will provide insights that will directly feed back into our next design 
cycle. The inclusion of design, practice, and investigation in design-based research allowed us to lay out 
our design, implement the design in a real class context, and empirically study the efficacy of our design. 
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II. DESIGNING FOR COGNITIVE PRESENCE 
A. Cognitive Presence 

Garrison, Anderson, and Archer [10] considered that all successful online discussions should promote 
cognitive presence. Cognitive presence describes the extent to which learners are able to construct 
knowledge through reflected discourses in a critical community of inquiry [10, 16], In supporting 
cognitive presence, two other elements are essential: social presence and teaching presence [10, 19, 20]. 
Social presence is defined as the level of a sense of belonging thought which participants project 
themselves as members of a community. Social presence is a feeling of togetherness and mutual 
awareness [21]. Teaching presence refers to designing and facilitating learning [11]. 


We designed and implemented the online discussion activity in a graduate level course, Training Methods 
and Strategies, at a university in a Southeast Asian country where English is the official language. There 
were 20 students enrolled in the class. Class discussion was an essential component in this course. The 
major purpose of carrying out class discussions was to promote student learning of various issues in the 
field of training methods. It was hoped that class discussions would stimulate active thinking, bring in 
multiple perspectives and provide students opportunities to apply what they had learned to interpreting 
various issues in training methods and strategies. 


The electronic discussion board was incorporated into the course as a medium for discussions. The online 
discussion overcomes some of the limitations of face-to -face class discussions. Its around-the-clock 
accessibility extends discussions beyond classrooms, and it allows one-to-one, one-to-many, and many- 
to-many interactions. Compared with spontaneous and transitory face-to-face class discussions, online 
discussions are text-based and more structured, providing students time to formulate thinking and 
compose postings, thus helping to promote student higher order learning [22, 8, 23, 24, 25]. The text- 
based feature of online discussions makes student thinking visible and leaves a permanent written record 
for student later review. “Text-based communication may actually be preferable to oral communication 
when the objective is higher-order cognitive learning” [19, p. 91], 


B. Analytical Framework for Design Elements 

The effective design requires a systematic approach. First and foremost, we need to identify design 
elements. What are essential design elements that govern the online discussion activity? Activity theory 
is the analytical tool we use for identifying design elements. Activity theory is a powerful framework in 
instructional design, especially in designing human-computer interactions in socio-cultural contexts [26]. 


Activity theory has its root in the work of Vygotsky. His pioneering work on learning in social-cultural 
contexts established the foundation for modem activity theory. Activity theory has the basic structure that 
contains (1) Subject; (2) Object; (3) Tools; (4) Rules; and (5) Roles. The subject is an individual or a 
group involved in the activity. The figure [27] below displays the basic structure of activity theory. The 
bold lines in Figure 1 represent mediation. Tools mediate between subject and object. By being 
incorporated into the process of human activity, tools help shape and alter human behaviors carried 
toward the object. Rules mediate actions of subjects endorsed by the community and regulate interactions 
within the group at both the individual level and social level. Roles refer to the division of labor and 
mediate among group members. Figure 1 displays the interconnectedness of the components in the basic 
structure of activity theory. 
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Figure 1. The Activity System [27] 


By using activity theory to analyze e-leaming communities, Elung and Chen [28] observed that object and 
rules exert great influences in shaping participants’ behaviors. The object determines the way the activity 
is carried out; the object constantly interacts with the subject and the community, providing them 
directions, validating the community’s efforts, and helping participants make sense of their actions. Rules 
determined the use of tools and roles. When rules are appropriately designed, from which the roles and 
tools are dictated, in-depth learning transactions would likely be actualized. In general, Hung and Chen 
[28] considered that object is the top level in the hierarchical structure of the Activity System. Rules are 
on a higher level of significance in the hierarchy compared with tools and roles. 


Based on their observations, Hung and Chen [28] redesigned the system by altering the activity structure. 
Figure 2 [28] displays the re-conceptualized activity structure. 

Outcomes of an Activity 


Object 



Shared Cognition 


Personal 

Understanding 

Figure 2. The Re-Conceptualized Activity System [22] 
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The online discussion activity reported in this paper was designed using the reconfigured activity 
structure. We placed great emphasis on designing the object/leaming outcome of the activity and the 
ground rules that dictated the discussion activity. 


C. Design of the Value-Added Object 

The object/leaming outcome was the ultimate goal that the discussion activity was to accomplish. To 
facilitate its accomplishment, it was essential that students make sense of the object and bought into its 
value. Without a clearly defined object, the online discussion tended to lose its cognitive essence and 
digressed to unfocused communications. Lave and Wenger [29] considered it imperative that participants 
had shared interests and made sense of their joint efforts. 


We considered that the object of the online discussion activity needed to be tied to the overall learning of 
the course. Research showed when students perceived the online discussion as irrelevant to their overall 
learning in the course, they did not engage in meaningful interactions [25], Students posted to satisfy the 
minimum course requirement. Their interactions were one way rather than two ways [2]. A study by 
Oliver and Shaw [30] found that students were actually playing a game of assessment; they posted 
messages to earn credits. The assessment rule motivated students to post messages; however, the rule 
failed to motivate students to engage cognitively in discussions. 


We aligned the object of this online activity to one of course assignments that students conducted research 
on a topic in training methods and strategies and presented their findings in class. This assignment was a 
group project. Students formed two-person groups. The online discussion was employed to scaffold 
students in completing this assignment. Students were required to post their topics, preliminary research, 
and their assertions on the discussion board. The class would then interact on these topics. Students 
would use input from discussions to revise and refine their research. Each group was required to 
incorporate the input from their peers into their final presentations. We hoped that this value-added 
object would have students perceive the activity in the context of a larger and more global picture. We 
designed this value-added object to facilitate formation of commonality and interdependence that were 
two essential features in building learning communities [29]. 


Commonality refers to learners’ shared goals and expectations for learning [31]. Commonality helps 
learners to make sense of their collaborative work. One major interest commonly shared by students was 
to complete course assignments and present high-quality work. We situated this online discussion in the 
context where it was used to help students perform well in a course assignment. In successful learning 
communities, all the participants had a stake in their investment of time and efforts and made sense of 
their investment with respect to each other [32]. It was expected that students would be bound to work 
together by this commonality. 


Interdependence emerges when learning requires differing competence levels of expertise and 
perspectives from participants. One team member’s success depends on his/her peer’s success [33]. 
“Interdependence is the mutually negotiated and accepted way of interacting among the parties with the 
recognition of each other’s perspective, interest, contribution and identity” [34, p.147]. The 
accomplishment of the desired object of this discussion activity required collective intelligence and 
collaborative work. The task was simply too difficult for a single person to accomplish. Students 
depended on the diverse expertise of their peers in pursuit of the desired learning outcome. 
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D. Design of Ground Rules 

1. Rules to Support Cognitive Presence 

Rules support the object of an activity. The purposes of this online discussion were to engage students 
cognitively and support strong cognitive presence. By participating in online discussions, students were 
expected to achieve better command of various issues in training methods and strategies. 


A literature review was conducted to identify problems that hinder student learning in online discussions. 
Frequently cited problems included non-participation [6, 35, 36], postings that did not reflect critical 
thinking skills [6, 25, 37], limited interactions among participants [30] and topic digression to unfocused 
talks [6, 38], These problems severely encumbered cognitive presence in online discussions. 


Research showed that student participation has positive association with learning outcomes in online 
activities [39]. Cognitive presence is actualized by student active participation. Nevertheless, student 
participation was a problem, even though the online discussion allowed students to have anytime-and- 
anywhere access [36, 40], A few students might post, while other students have been comfortable in 
being “barkers,” reading rather than contributing to the discussion [41], When some students did post, 
they seldom received responses [42], Therefore, student interactions were rather limited [30], 


It remains a challenge to sustain cognitive presence in online discussions. Student discussions often 
lacked critical components. Student postings were often shallow, superficial, subjective and at times 
naive, showing little evidence of critical thinking [43]. Rather than critically evaluating and synthesizing 
course materials, students mainly exchange personal experiences and retold course materials [25]. 


Another challenge in promoting cognitive presence is to keep online discussions on topic. Samuel [44] 
considered this issue a key challenge. Students exerted more control over their learning when they were 
online than they did in face-to-face learning. In addition, threaded discussions allowed students to follow 
various threads. As a result, online discussions tended to lose their original focus and digressed to 
irrelevant talks. Once discussions wandered off track, it would take a long time for the instructor to get 
discussions under control [38]. 


We took a proactive approach in designing rules. Rules were designed to promote cognitive presence 
while avoiding the pitfalls that encumbered student learning in online discussions. When laying down 
each rule, we identified rationales underlying the rule. Questions that were constantly in our minds were: 
In what way did the rule support cognitive presence in online discussions? How should the rule be 
designed to avoid or alleviate the frequently cited problems in the literature? Table 1 displays our rules. 


Rules 

Rationale 

Start dates 

Level playing field 

Cut off dates 

Facilitate student planning 

Minimal number of posts — You need to 

Increase number of posts 

comment on at least two other groups. 

Critical mass 
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Support your arguments with evidence 
(established theories, empirical data, thought 
experiments, etc.). 

Prevent garbage messages 

Promote meaningful cognitive engagement 

Increase quality of posts 

Keep one point per short message 

Increase clarity of the structure of messages 

If no one answers your posting, you can send 
invitations to three students for responses. 

Legitimacy for response 
Shortened waiting time 

You are not allowed to post before the 
second deadline. 

Useful delay for lurking 
Increase thoughtful postings 

You are encouraged to build on existing 
ideas by quoting and paraphrasing other 
people’s messages. 

Build on each other’s ideas 
Virtual “body language” 

Expand perspectives of the discussed issues 

Reduce tension between active and lurking 
participants 

You must always reply to comments to your 
posts 

Increase interaction 

Greater ownership of discussion 

If you have nothing more to add, wrap it up 
nicely with a concise summary. 

Every message matters 

Proper closure 

Clarify more ideas 

Increase student synthesis skill 

Open new directions for discussion 


Table 1. Ground Rules to Sustain Cognitive Presence 


2. Social Presence to Support Cognitive Presence 

We believe that learning is inseparable from learners’ emotions. Within a learning community, learners 
should feel free to take risks to achieve the desired learning outcome. While promoting cognitive 
presence, we set up rules in facilitating group dynamic interactions — social presence. These rules were to 
facilitate building a healthy community of inquiry and developed a student-centered discussion 
environment. 


This is a place to celebrate our ignorance, imperfection, confusion and struggles. 

Keep the discussion informal 

You are strongly encouraged to try ideas that you are not yet certain of (dare to risk). 

Keep messages short and sweet. 

Be constructive! It is always possible to disagree 

Do not spare your compliments. Be encouraging and constructive. 

Questions can be powerful contributions. Sometimes, you don’t even have to agree or disagree. 
Play the devil’s advocate, but do signify your intent in the message. 

Do not feel distressed if the lecturer does not comment on your messages. There can be many 
reasons. For example, the lecturer can be overwhelmed, he/he does not have anything more to add, 
he/she is waiting for others to try out ideas first, etc. 
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Our value-added object was also aimed at facilitating the sense of togetherness and mutual awareness of 
the students in relation to each other. Bound by the shared goal and the need for diverse competency 
levels of expertise and perspectives in completing the task, students were expected to collaboratively 
maintain and nurture the formation of the community of inquiry. 


Social presence needed to be promoted and nurtured throughout the course. This was a course in which 
students met face-to-face. The instructor of the course made efforts in facilitating the formation of the 
community of inquiry throughout the course by allowing students to be the major force in directing their 
own learning. Students were encouraged to take risks and challenge each other as well as experts, which 
played a significant role in building relationships among students and increased student comfort level in 
functioning in the online community of inquiry. 


E. Teaching Presence — Teachers as Designers and Facilitators 

Teaching presence is binding element in creating a community of inquiry. Fundamentally, cognitive 
presence and social presence largely depends on the presence of a teacher [20]. The text-based 
environment differs from face-to-face environment in that it has low teacher immediacy. Teachers and 
students are separated physically. Teachers are not always available when needed. Therefore it is more 
important to promote student autonomy in online discussions. Using activity theory as the framework, we 
focused our efforts on designing the valued-added object, rules and social norms. Bound by the shared 
object and guided by the rules and social norms, we believed, spontaneous facilitation among students 
would emerge. Students would be able to take responsibilities for their own learning. We decided we 
would move ourselves to the side and invite students to take the center stage. Students would be the ones 
to take charge, initiating dialogues, keeping the discussion focused, stimulating discussions, encouraging 
and supporting each other and resolving group conflicts. Although we would monitor the discussion 
closely, we had decided that we would not intervene unless absolutely necessary. 


III. RESEARCH METHODS 

Content analysis was used to measure cognitive presence in student online discussions. The unit for 
analysis was based on the message level. Research confirmed that content analysis based on the message 
level was the reliable method to assess cognitive presence in online discussions [1 1, 45]. 


Garrison, Anderson, and Archer [11] used content analysis to measure cognitive presence and developed 
a coding system that was applied in research on cognitive presence in online discussions [3, 45], 
According to their analysis, Cognitive presence progresses and develops through four stages: triggering, 
exploration, integration, and resolution (table 2). 


Category 

Indicators 

Triggering 

Recognizing the problems 
Sense of puzzlement 

Exploration 

Divergence within online community 
Divergence within single message 
Suggestions for consideration 
Brainstorming 
Leaps to conclusions 
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Integration 

Convergence among group members 
Convergence within a single message 
Connecting ideas and synthesis 
Creating Solutions 

Resolution 

Various application to real world 
Testing solutions 
Defending solutions 


Table 2. Cognitive Presence [11] 


In this study, cognitive presence was operationalized through three stages since the resolution phase was 
not applicable in our case. Olszewski [46] pointed out that “it may not be appropriate for resolution to be 
the outcome of a single course” [46, p. 117]. The students in this course would not test their solutions in 
real world situations. The final product of the discussion was a class presentation. Please note the 
definition of resolution and do not confuse it with that of group consensus that is included at the 
integration stage. All the student postings were archived and analyzed based on message-level units. 
Messages were categorized into the three phases of cognitive presence: triggering, exploration, and 
integration. 


Inter-rater reliability was established in three phases. First, 10% of the messages were randomly selected. 
The two researchers independently coded the data. Upon completion, the two researchers discussed the 
results of the coding. Discrepancies in the coding were identified and the coding rule was refined. For 
example, if a message defended a previously stated position, it should be coded as “integration.” This 
phase was repeated until 1 00% agreement was reached. 


Second, one researcher completed the coding of the entire data set. Third, another 10% of the remaining 
messages were then randomly selected for the other researcher to conduct coding. The reliability of data 
coding of these randomly selected messages was .92. 


Upon completion of their class presentation, students were asked to reflect upon their experiences with 
online discussions by responding to a list of questions. These questions explored student perceptions 
regarding online discussions. Student responses were examined to provide insights on the efficacy of our 
design. 


IV. FINDINGS 

A. Strong Cognitive Presence in the Online Discussion 

The students formed two-member groups to investigate an issue in training methods and strategies. They 
were required to post their preliminary research and assertions on the discussion board. The class 
responded to the postings. The groups would then use responses on the discussion board in preparing 
class presentations. The time span for the discussion forum was 20 days. 


Student enthusiasm for participation was impressive. In total, this discussion generated 267 messages. 
The quantity and quality of student postings were indicative of student deep involvement in the 
discussion. Although students were required to respond to two other group’s postings, student 
contributions far exceeded the required participation. Students reflected that: 
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Personally, my participation level went beyond the stipulated minimum 2 posts as I actually 
read all the postings and responded to things I do not agree with — such as “argument is a 
character trait.” 

I was equally excited by the amount of quality discussions that were posted. The numerous 
viewpoints and concerns that flowed out of the thread are interesting and enlightening. This 
may be due to the interesting assertions that were being posted. All seem eager to learn about 
the assertions and help in refining them. Overall, it is a great learning experience. 


Cognitive presence was strong in the discussion. Findings are displayed in table 3. Only 6% of messages 
fell into the category of “Others”. Messages in this category were acknowledgements and appreciations of 
other student contributions. A couple of messages were sent for humorous reasons. 


Category 

Number 

Percent 

Triggering 

58 

22% 

Exploration 

111 

41% 

Integration 

83 

31% 

Others 

15 

6% 


Table 3. Cognitive Presence in the Online Discussion 


The following sections detail student participation in each phase of cognitive presence. 


1. Triggering 

Triggering is the initial stage of cognitive presence. In the triggering phase, 22% of the messages were 
generated. 


The initial stage of the discussion was marked by student inquiries and elementary clarifications. After 
all the groups posted their assertions and rationales for their investigations, questions started to pour in. 
Students posted questions to seek clarification or challenge the rationale of assertions. Questions 
triggered responses. Responses triggered more questions. Sometimes, questions triggered questions. 


One group posted their assertion regarding the role of gaming in learning: 

Game-based learning is an effective instructional strategy for learning outcomes that are higher 
order learning, for example, problem solving”. This assertion triggered a series of questions. 


There were questions to seek more clear definitions: 

By game-based, are you referring to game in general, which connotes fun? If you are, then I have 
experienced being taught something in a fun (or game in your term) way but it was to learn some 
very elementary materials. 


Some questions challenged the rationale of the assertion: 
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Plow do you assess effectiveness of game-based learning therefore quantifying the pros and cons? 

Is it suitable? If a Subject Matter Expert wants a game-based environment, the first things I’ll look 
at would be: (1) cost; (2) expertise; (3) justification. 


There were also inquiries that demanded more evidence: 

Xiang, could you show some evidence or support from learning science that illustrates that gaming 
helps in higher order thinking skills (in fact based on your response is that gaming can be helpful 
for lower-order learning outcome as well?) Then we have a clearer idea to assess between yours 
and Kang’s assertion. 


Poscente and Fahy [47] considered that strategic initial messages had potential to trigger cognitively 
engaged discussions. They termed questions that invited discussions as true triggers. These questions 
were open-minded and solicited responses from participants. Questions forced groups to dig deeper into 
the issues, re-think the rationale of their assertions, and decided on the direction where to take their 
investigation. Questioning helped the groups to identify logical gaps, one-sided views and new 
perspectives. In the process to construct clarifications and explanations, students collected and examined 
evidence and performed analysis and synthesis. Karron Lewis [48] considered that questioning played an 
essential role in systematic investigations in any field. Questions are used to evaluate reasons for the 
investigation, to steer the investigation, and to examine results. 


2. Exploration 

At the exploration phase, students externalized their thoughts, broadening their views of the investigated 
issues. In total, 41% of the messages were generated in this phase. Through collaborative exploration, 
multiple perspectives emerged. Exploration was the essential process of socially constructing knowledge. 
Problems in the real world rarely have one correct solution path. The exploration phase provided students 
opportunities to evaluate various solution pathways, extending and enhancing student understanding of 
their investigations. As the discussion progressed, the discourse moved from surface level processing to 
in-depth level processing. Students externalized their viewpoints, compared their thoughts with that of 
their peers, and evaluated various claims and arguments. Contradictory ideas, personal opinions, and 
thoughts that were not quite mature were expressed. One group posted the assertion: “Authentic learning 
approach is NOT suitable for secondary and junior colleges in our country.” In the process of 
exploration, students discussed various aspects of the issue in the context of their country. Students first 
brought up confounding issues in various learning modules: 


How to give grades in this case? After going through all these, how do we make sure that they are 
really the kind of “product” that we want? I have seen students doing a great project, but unable to 
answer simple questions. . . .Why? Because they didn’t internalize the knowledge. They 
forget. . .strange, isn’t it? On the other hand, students who score A are unable to produce good 
projects. (Of course there are exceptions, but how many?) 


Some students pointed out that teachers were not motivated to apply authentic learning in classrooms 
because they were pressured to prepare students for tests: 


I hate drill and practice and personally like authentic learning methods. The objective of getting 
good grades for the kids is set by MOE (Ministry of Education). So teachers are pressured to 
ensure they get the grades, though I’m not sure what’ll happen to teachers who chose a different 
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approach form the norm - kinda reminds me of the movies Dead Poets Society and Mona Lisa 
Smile. 


The issue of the hiring system in the country was also raised. Students realized that the shift in assessment 
should also apply to the job market. 


The current practice is to look at their grades in schools which is the traditional assessment 
paradigm. Just look at the public sector, when you apply for a job, they require you to fill in even 
your primary grades. Isn’t this product-based rather than process-based? Although an interview 
assesses in learning process, the key to getting the interview is the product, the grades. I feel if 
there is to be a shift in assessment, the working world should also be educated on it; if not, they 
will be looking at the wrong things when assessing potential candidates. 


Through communicating diverging thoughts, students started to view the issue through wider lenses. One 
student posted that the extremity of the assertion made her uncomfortable: “It is like banning marriage for 
all (akin to secondary and JC students) because we can’t measure for sure that married couple are truly, 
truly, truly blissfully married (akin to the assessment mode is not changed)”. Students suggested that the 
group examine and readjust their assertion: 


Just want to point out your original assertion did not include the point that due to the assessment 
methods and time constraints, authentic learning has no place in secondary school and junior 
colleges, although you did try to explain this point in the justification. Maybe you want to refine 
the assertion to reflect this point. 


Rather than viewing the issues from a particular perspective, students embraced multiple perspectives. In 
the engaging, and often heated discourse, students communicated, negotiated, and reflected. 


In terms of refining my assertion, I have learned that what I believed to be co mm on terms are 
not so co mm on after all. Different people view technical terms differently. In the process of 
refining the assertion, the questions and comments made probe me to think deeper into the issue 
and help in looking at things with a wider len. I could sense that everyone who helped in 
questioning the assertion is genuinely concerned and interested in the assertion. 


3. Integration 

Integration is the challenging phase of cognitive presence. In the integration phase, 31% of the messages 
were generated. 


The integration phase was the process in which students reached consensus through collaborative 
inquiries. Ideas and thoughts started to converge. Students interacted intensively, making references to 
each other’s messages, and building upon each other’s contributions. It is a process where students 
started to internalize various perspectives. 


Student messages became lengthy and substantiated. Students elaborated on their thoughts and debated at 
a deeper level. More importantly, students supported their thoughts and arguments with evidence. 
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If you want to place OJT (On-Job-Training) into any of the learning theories that we know of, I 
would place it under social constructivism. It is about working together with others to learn. The 
environment or the social circle is an important component. 


If you place OJT in Gagne’s thinking on domain of learning, I will say that OJT is applicable for 
all, namely, psychomotor, verbal, intellectual skills cognitive strategies and attitude (Note that they 
are not listed in term of applicability, simply based on what I could recall... here...). In term of 
concept learning, OJT should be classified as the best example related to their workflow. 


Well, in term of whether OJT is a “simple good learning from the ‘master,’” I would say yes if 
looked simplistically. It should be noted that OJT tends to be a component of apprenticeship, 
therefore if you look at the apprenticeship model, you could say it is learning from the “master.” 
But our point is that it should not be from the “master” but through a social system whereby they 
learn from capable peers. 


Reasoning with evidence required students to collect, examine, and apply evidence in supporting or 
disputing claims and arguments. Students identified information resources, assessed the relevance of the 
information and connected the information with their discourses. Evidence came from a wide range of 
information sources: course readings, journal articles, web resources, government documents, PowerPoint 
presentations, workshops, and videos. 


As the integration phase progressed, shared cognition emerged. For example, one group posted their 
assertion: Computer gaming can impair students ’ ability to learn and perform. In discussions, students 
explored the addiction to excessive gaming, the gender issue in gaming, the age factor, and the potential 
of gaming in learning. Postings from other groups had the group realize that they should take a balanced 
approach toward computer gaming. 


I also agree with Hong that originally that our assertion is just “computer gaming can impair 
students’ ability to learn and perform”. But things seem to evolve depending on inputs from ALL 
participants. 


Hong has highlighted himself as a good example to have combined/coexisted with “the evils” of 
gaming, that may have consumed a lot of time, but nevertheless, you have achieved good results 
in life too. Perhaps the way to life is balanced activities. 


I disagree on “Addiction to gaming is simply a manifestation of a person’s willpower.” I believe 
a lack of parental control, guidance and supervision plays a more critical role/condition to 
development of addiction of computer games in the gamers’ formative years. 


As the result of the discussion, the group realigned their assertion: Excessive time spent on entertainment 
complex online games takes away time for school lessons. 


Data showed that all the groups integrated elements from the discussion into their class presentations, 
thoughts emerged in the discussions, research evidence was provided by peer groups, and suggestions 
were followed for refining assertions. 
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V. REFLECTION UPON OUR DESIGN 

In this section, we’ll elaborate and reflect on the efficacy of our design based on the findings of our 
investigation. 


A. Value-added Object to Support Cognitive Presence 

Critical thinking is both a process and a product. As a learning outcome, critical thinking is best evaluated 
through course assignments. More importantly, acquiring critical thinking skills can be supported by the 
use of a tool and facilitated by better understanding of this process [11]. 


The object for this online discussion activity involved both product and process. The tangible product for 
this online learning activity was student presentation. The online discussion activity was designed to 
prepare students for their class presentations. From the process perspective, the object was to use the 
discussion board as a tool to engage students cognitively and enhance their understanding of investigated 
issues. 



Learning Outcome 
Class 

Presentations 


Student Posting 
Investigated Issue 



Online 

Discussions 

Cognitive 

Presence 



Figure 3. The Multi-phased Inquiry Cycle 


Instead of designing the online discussion as a stand-alone activity, we situated this online activity in a 
multi-phased inquiry cycle (figure 3). The online activity was designed to scaffold students in completing 
their course assignment. We considered it essential for the online discussion to result in a tangible 
product, be it a presentation, a paper, or a project. The multi-phased inquiry cycle facilitated student 
articulation and reflection of their learning. In the process of preparing for class presentations, students 
related the discussion to the assigned project, evaluating strengths and weaknesses of their arguments as 
well as those of their peers, synthesizing and integrating discussion components into the class 
presentation. 


The inquiry cycle added reflective elaboration as a dimension to student discourse. The permanent written 
record of the online discussion allowed students to view and reflect upon the discourse, consolidating 
various thoughts and arguments into the class presentation. This study showed that the inquiry cycle was 
effective in promoting the online discussion. The tangible product was an essential element to complete 
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the inquiry cycle. Without a tangible product, the inquiry cycle would be incomplete, missing a critical 
component for reflective elaboration. 


B. Ground Rules to Sustain Cognitive Presence 

The ground rules we designed were effective in enhancing cognitive presence in each phase of the 
discussion. Frequent problems such as non-participation, superficial messages and topic digression did 
not occur in this online activity. Comparing with previous studies [45, 49], students in this study 
generated more messages in the integration phase. There were only 6% of postings classified in the 
category of “others,” which signified the intensity of student concentration on the topics discussed. 


Student performance in the discussion was beyond our expectations. Although we required that each 
student commented at least on two postings from other groups, the majority of students did more than 
that. There was active interaction across groups. The strong cognitive presence in the discussion showed 
that students valued this online discussion as an important activity in their learning. 


In reflecting upon our rules, we realized that our rules corresponded to each phase of cognitive presence 
(table 4). This finding has a great implication for setting and refining rules for online discussions. With 
clear ideas about what to promote in each phase, we can design rules to solve problems in a particular 
phase of cognitive presence. For example, questions are effective triggers. To encourage students to ask 
questions in the triggering stage, students might be required to choose controversial topics, rather than 
topics that are universally agreed upon. In the exploration phase, we should work on rules that facilitate 
free articulation of thoughts and perspectives. To move students to the integration phase, efforts should be 
spent in setting rules to help students connect ideas and transact learning to a higher cognitive level. 


Rules Supporting Cognitive Presence 

Triggering 

Exploration 

Integration 

Start dates 
Cut off dates 

* 



Minimal number of posts - You need to comment 
on at least two other groups. 

* 

* 


Support your arguments with evidences 
(established theories, empirical data, thought 
experiments, etc.). 


* 

* 

Keep one point per short message 

* 

* 

* 

If no one answers your posting, you can send 
invitations to three students for responses. 

* 



You are not allowed to post before the second 
deadline. 

* 

* 


You are encouraged to build on existing ideas by 
quoting and paraphrasing other people’s messages. 


* 

* 

Y ou must always reply to comments to your posts 

* 

* 

* 
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If you have nothing more to add, wrap it up nicely 
with a concise summary. 



* 


Table 4. Rules Corresponding to Phases of Cognitive Presence 


In analysis of rules with respect to the findings, we considered we benefited from several design 
principles we followed: 


1. Initial Postings Are Essential 

Initial postings set the standard and quality for later postings. Cognitive presence needed to be built into 
initial postings. For initial postings, we required students to post their assertions regarding an issue on 
training methods as well as justifications for assertions based on research evidence. Students needed to 
do some initial research in framing their assertions. They needed to access, analyze and evaluate relevant 
research in formulating their assertions. Assertions as such solicited higher level of cognitive responses 
and directed participants to reason with evidence in their responses. 


2, Rules Need to be Made as Specific as Possible 

While making rules, we attended to specifics. Only when the rules were made specific, could students 
know clearly the expectations of them and follow the rules accordingly. For example, what counted for 
reason with evidence! We specified the evidence as “established theories, empirical data, thoughts 
experiments, and personal experiences.” Examples could help illustrate rules and make rules easier to 
follow. Since all the student messages were archived on the discussion board, it should be fairly easy to 
select examples to be used in future classes. 


3. Flexibility Needs to Be Built Into Rules 

Rules and flexibility were not exclusive to each other. In making rules, we infused a level of flexibility by 
providing students options. For example, students were required to respond to every message sent to 
him/her. This rule ensured student participation and interaction. However, students might have run out of 
what needed to be said at times. In this case, we allowed students to wrap up the topic to summarize the 
thread. This rule worked well. It kept the discussion flow while allowing students to operate their 
cognitive skills such as analysis and synthesis. 


C. Social Presence to Support Cognitive Presence 

Social presence in the discussion contributes to cognitive presence. Social presence helps students project 
themselves as members in this online learning community. Learning is affective and emotional. Social 
presence has great influence on learning outcomes [50]. Learners learn best in an environment where 
they have a sense of connection and feel free to take risks. 


We set rules to promote and support social presence. These rules helped to create a democratic and 
relaxing environment. Students voiced their thoughts freely. For example, in the exploration phase, ideas 
and thoughts were often not quite mature. However, students were willing to take risks, testing their 
thoughts and arguments, which indicate that students were comfortable in interacting with each other. 
Students questioned, challenged and debated. Uneasiness and offenses found no place in this community 
of inquiry. The discussion environment was democratic and student-centered where cognitive presence 
was accomplished through collaborative critical inquires. 
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D. Teaching Presence to Support Cognitive Presence — Faculty as 
Facilitators vs. Students as Facilitators 

Our design was successful in promoting student autonomy in the online discussion. Elung and Chen [28] 
hypothesized that roles were dictated by the object and rules. Bounded by the object and rules, students 
took the role of facilitators in this discussion. The instructor did not send any messages to urge students 
to post. Throughout the discussion, the instructor only intervened twice to clarify concepts. Students 
were the ones in control, taking initiatives, debating on differing thoughts, encouraging and supporting 
each other, keeping the discussion on topic, and moving the discussion towards the desired object. 
Student postings centered on the three phases of cognitive presence. For example, one student was not 
sure whether her posting was on the track, she started her message apologizing: “Forgive me if I have 
side-tracked from the discussion.” Another student responded immediately: “There is nothing to forgive, 
as these are all done in just good-nature, isn’t it?” This example showed that students were conscious of 
keeping the discussion on topic as well as being willing to assume the facilitators’ role to encourage and 
support each other. Compliments such as “Fantastic,” “Good tips” and other praise abounded in students’ 
replies. 


While reflecting upon our design, we considered it essential to clearly articulate the object and rules of the 
discussion activity. The object and rules provided the structure and guidance for student learning. By 
moving to the side in the process of the discussion, we offered students opportunities to take the challenge 
in directing their own learning. Although we were not at the center of the stage, we monitored the 
discussion process closely and intervened when there was a need so students knew that we were there and 
available for help. 


VI. RECOMMENDATIONS FOR FUTURE STUDIES 

1. One of the purposes of design-based research is to advance theory that transcends a particular set of 
contexts [51]. In our research, we used the refined Activity System by Hung and Chen [28] as the design 
framework and achieved desired learning outcomes. To validate and confirm the refined Activity System, 
we recommend that the study be replicated in courses of different levels (e.g., graduates vs. 
undergraduates); with diverse subject matters (e.g., science vs. social sciences); and across various 
learning modes (e.g., distance learning, blended learning, and face-to-face learning). 

2. Throughout the study, we observed that there was an interactive relationship between the rules we set 
and the online learning management system we used. The limitations imposed by the learning system 
compelled us to establish certain rules. For example, we required that students quoted their peers’ 
messages because the system did not provide a convenient way to make references to other messages. We 
required students to acknowledge the messages they received because the system did not have an 
automatic acknowledgement function. We recommend that future studies focus on designing a new set of 
tools with expanded affordances in facilitating effective online discussions. 


3. Traditionally, instructors largely assumed the role of facilitators in discussions. Online learning 
environments opened opportunities for students and faculty to share the role of facilitators. In this online 
discussion, spontaneous facilitation occurred among participants. Students were the ones who presented 
questions, kept the discussion focused, summarized the discussion, and confirmed understanding. 
Students played an essential role in successful online discussions, not only as participants, but also as 
facilitators. Some studies suggested that it was beneficial to assign facilitators’ roles to students in online 
discussions [52, 53], For instance, students were assigned the roles of group leaders [52], devil’s 
advocates, eternal optimists and pessimists [54]. We recommend that comparative studies be conducted to 
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investigate types of student involvement as facilitators in terms of student motivation and cognitive 
presence in the online discussion. 


VII. CONCLUSIONS 

This design-based research allowed us to design an online discussion activity, implement it, and use the 
real classroom as a natural laboratory to observe, examine, and revise the design. 


Our design embodied our hypothesis that the object and rules in an activity significantly influenced the 
success of the activity. Our empirical investigation validated our hypothesis and demonstrated that our 
design was effective in promoting cognitive presence in the online discussion. 


The fundamental purpose of this design-based research is to improve teaching and learning in university 
courses with online technology. With the trend of integrating online technology into various learning 
settings to enhance the learning of diverse student populations, university instructors face great challenge 
in teaching in new online environments. It is our hope that this study provides a design framework that 
can be adapted, modified, and applied by practitioners for online learning activities in university courses. 


VIII. ACKNOWLEDGEMENTS 

The authors of this paper wish to tha nk all the students who participated in the online discussions reported 
in this study. 


IX. REFERENCES 

1. Aviv, R. Educational performance of ALN via content analysis. Asynchronous Learning Networks 
4(2): 53-72, 2000. 

2. Hara, N., C. J. Bonk & C. Angeli. Content analysis of online discussion in an applied educational 
psychology course. Instructional Science 28: 115-152, 2000. 

3. Meyer, K. A. Face-to-face versus threaded discussions: The role of time and higher-order thinking. 
Asynchronous Learning Networks 7(3): 55-65, 2003. 

4. Thomas, M. J. W. Learning within incoherent structures: the space of online discussion forums. 
Computer Assisted Learning 18(3): 351-366, 2002. 

5. Wu, D. & S. R. Hilt/. Predicting learning from asynchronous online discussions. Asynchronous 
Learning Networks 8(2): 139-152, 2004. 

6. Klemm, W. R. What’s wrong with on-line discussions and how to fix it. Paper presented at WebNet 
World Conference, ERIC Document Reproduction Service No. ED 448 755, 2000. 

7. Valle, R., S. Oncu, N. F. Koksal, P. Alford & T. M. Duffy. Effects of online cognitive facilitation 
on student learning. Paper presented at the annual meeting of Association for Educational 
Communications and Technology, Chicago, IL. ERIC Document Reproduction Service No. ED 
484986, 2004. 

8. Black, A. The use of asynchronous discussion: Creating a text of talk. Contemporary Issues in 
Technology and Teacher Education 5(1): 5-24, 2005. 

9. Knowlton, D., H. M. Knowlton & C. Davis. The whys and hows of online discussion. Syllabus 
13(10): 54-56,2000. 

10. Northover, M. Online discussion boards — Friend or foe? Proceedings of Australian Society for 
Computers in Learning in Tertiary Education, Melbourne, Australia, pp. 169-177, 2001. 


Journal of Asynchronous Learning Networks, Volume 12: Issue 3-4 


174 


Essential Elements in Designing Online Discussions to 
Promote Cognitive Presence — A Practical Experience 


1 1 . Garrison, D. R., T. Anderson & W. Archer. Critical thinking, cognitive presence, and computer 
conferencing in distance education. The American Journal of Distance Education 15(1): 7-23, 2001. 

12. Bell, P. On the theoretical breath of design-based research in education. Educational Psychologist 
39(4): 243-253, 2004. 

13. Wang, F. & M. J. Hanna tin. Designed-based research and technology-enhanced learning 
environment. Educational Technology Research and Development 53(4): 5-23, 2005. 

14. Brown, A. L. Design experiments: Theoretical and methodological challenges in creating complex 
interventions in classroom settings. Journal of the Learning Sciences 2(2): 141-178, 1992. 

15. Collins, A. Toward a design science of education. In E. Scanlon & T. O’Shea (Eds), New directions 
in educational technology, 15-22. New York: Springer- Verlag, 1992. 

16. Joseph, D. The Practice of design-based research: Uncovering the Interplay between design, 
research, and the real-world context. Educational Psychologist 39(4): 235-242, 2004. 

17. Sandoval, W. A. Developing learning theory by refining conjectures embodied in educational 
designs. Educational Psychologist 39(4): 213-223, 2004. 

18. Sandoval, W. A. & P. Bell. Design-based research methods for studying learning in context: 
Introduction. Educational Psychologist 39(4): 199-201, 2004. 

19. Garrison, D. R., T. Anderson & W. Archer. Critical thinking, cognitive presence, and computer 
conferencing in distance education. The American Journal of Distance Education 15(1): 7-23, 2001. 

20. Garrison, D. R., T. Anderson & W. Archer. (2000). Critical inquiry in a text-based environment: 
Computer conferencing in higher education. Internet and Higher Education, 2 (2-3), 87-105. 

21. Hamman, B. Two voices: Social presence, participation, and credibility in online news. Unpublished 
master’s thesis, University of Missouri-Columbia, 2006. 

22. Bailey, Y. S. & V. H. Wright. Innovative uses of threaded discussion groups. Paper presented at the 
annual meeting of the Mid South Educational Research Association, Bowling Green, KY. ERIC 
Document Reproduction Service No. ED 446 716, 2000. 

23. King, K. P. Educators revitalize the classroom “Bulletin Board”: A Case study of the influence of 
online dialogue on face-to-face classes from an adult learning perspective. Research on Computing in 
Education 33(4): 337-355, 2001. 

24. Rourke, L. & T. Anderson. Exploring social presence in computer conferencing. Interactive 
Learning Research 13(3): 257-273, 2002. 

25. Biesenbach-Lucas, S. Asynchronous discussion groups in teacher training classes: perceptions of 
native and non-native students. Asynchronous Learning Networks 7(3): 24-46, 2003. 

26. Jonnasen, D. H. Activity theory as a framework for designing constructivist learning environments. 
Educational Technology Research and Development 47(1): 61-79, 1999. 

27. Cole, M. & Y. Engestrom. A cultural-historical approach to distributed cognition. In G. Salomon 
(Ed.). Distributed cognitions: Psychological and educational considerations. Cambridge: Cambridge 
University Press, 1991. 

28. Hung, W. L. & D. Chen. Situated cognition, Vygotskian thought, and learning from the 
communities of practice perspective: Implications for the design of web-based e-learning. 
Educational Media International 38(1): 3-12, 2001. 

29. Lave, J. & E. Wenger. Situated learning: Legitimate peripheral participation. Cambridge University 
Press, Cambridge, 1991. 

30. Oliver, M. & G. P. Shaw. Asynchronous discussion in support of medical education. Asynchronous 
Learning Networks 7(1): 56-67, 2003. 

31. Rovai, A. P. Building Sense of Community at a Distance. The International Review of Research in 
Open and Distance Learning 3(1): 2002. http://www.irrodl.org/index.php/irrodl/article/view/79/152 . 

32. Wenger, E. Communities of Practice: Learning, Meaning, and Identity. Cambridge University 
Press: Cambridge, 1998. 

33. Kreijns, K., P. A. Kirschner, & W. Jochems. Identifying the pitfalls for social interaction in 
computer-supported collaborative learning environments: a review of the research. Computers in 
Human Behavior 19(3): 335-353, 2003. 


Journal of Asynchronous Learning Networks, Volume 12: Issue 3-4 


175 


Essential Elements in Designing Online Discussions to 
Promote Cognitive Presence — A Practical Experience 


34. Bouwen, R. & T. Taillieu. Multi-party collaboration as social learning for interdependence: 
Developing relational knowing for sustainable natural resource management. Community & Applied 
Social Psychology 14(3): 137-153, 2004. 

35. Klemm, W. R Eight ways to get students more engaged in online conferences. T.H.E Journal 26(1): 
62-64, 1998. 

36. Shaw, G. P. & W. Pieter, The use of asynchronous learning networks in nutrition education: Student 
attitude, experiences, and performance. Asynchronous Learning Networks 4(1): 40-51, 2000. 

37. McLoughlin, C. & J. Luca. Cognitive engagement and higher order thinking through computer 
conferencing: We know why but do we know how? Flexible Futures in Tertiary Teaching. In A. 
Herrmann and M.M. Kulski (Eds.), Proceedings of the 9 th Annual Teaching Learning Forum. Perth: 
Curtin University of Technology, 2000. http://lsn.curtin.edu.au/tlf/tlf2000/contents.html . 

38. Beaudin, B. P. Keeping online asynchronous discussions on topic. Asynchronous Learning Networks, 
3(2): 41-53, 1999. 

39. Webb, E., A. Jones, P. Barker & P. V. Schaik. Using e-learning dialogues in higher education. 
Innovations in Education and Teaching International 41(1): 93-103, 2004. 

40. William, S. & C. Pury. Student attitudes toward and participation in electronic discussion. 
International Journal of Educational Technology 3(1): 2002. 

41. Goodell, J. & B. Yusko. Overcoming barriers to student participation in online discussions. 
Contemporary Issues in Technology and Teacher Education 5(1): 77-92, 2005. 

42. Collett, D., H. Kanuka, J. Blanchette & C. Goodale. Learning Technologies in Distance 
Education. Edmonton, AB: University of Alberta, 1999. 

43. Angeli, C., N. Valanides & C. J. Bonk. Communication in a web-based conferencing system: The 
quality of computer-mediated interactions. British Journal of Educational Technology 34(1): 31-43, 
2003. 

44. Samuel, A. 10 ways to keep online dialogue on topic. Social Signal blog, 2005. 
http://www.socialsignal.com/blog/alexandra-samueE10-wavs-to-keep-online-dialogue-on-topic . 

45. Vaughan, N. & D. R. Garrison. Creating cognitive presence in a blended faculty development 
community. Internet and Higher Education 8(1): 1-12, 2005. 

46. Olszewski, L. Deep learning and cognitive presence in collaborative web-based learning 
environment: Student and instructor perspectives. Unpublished master’s thesis, Athabasca University, 
Athabasca, Alberta, 2006. 

47. Poscente, K. R. & P. J. Fahy. Investigating triggers in CMC text transcripts. The International 
Review of Research in Open and Distance Learning 4(2): 2003. Retrieved September 30, 2006 from: 
http ://www. irrodl.org/index.php/irrodl/article/view/ 1 4 1 122 1 . 

48. Lewis, K. G. Developing questioning skills. Center for Teaching Effectiveness, University of Texas 
at Austin, 2002. http://www.utexas.edu/academic/cte/sourcebook/questioning.html . 

49. Garrison, D. R., T. Anderson & W. Archer. Critical thinking, cognitive presence and computer 
conferencing in distance education. American Journal of Distance Education 15(1): 2001. 

50. Jolivette, B. J. Social presence and its relevancy to cognitive and affective learning in an 
asynchronous distance-learning environment: A Preliminary literature review. Paper presented at the 
annual meeting of Academy of Human Resource Development International Research Conference, 
Columbus, Ohio, 2006. 

51. Barab, S. & K. Squire. Design-based research: Putting a stake in the ground. The Journal of the 
Learning Sciences 13(1): 1-14, 2004. 

52. Heflich, D. & L. Putney. Reflections of reality: Online conversation in a teacher education seminar. 
Intimacy and reflection, online conversation in a practicum seminar. Computing in Teacher 
Education 17(3): 10-17,2001. 

53. Henri, F. Computer conferencing and content analysis. In A.R. Kaye (Ed.), Collaborative Learning 
through Computer Conferencing: The Najaden Papers, 115-136. Berlin, New York: Springer-Verlag, 
1992. 


Journal of Asynchronous Learning Networks, Volume 12: Issue 3-4 


176 


Essential Elements in Designing Online Discussions to 
Promote Cognitive Presence — A Practical Experience 


54. Murray, B. Reinventing class discussion online, Internet and Psychology 31(4): 2000. 
http://www.apa.org/monitor/aprOO/reinventing.html . 

X. ABOUT THE AUTHORS 

Dr. Yu-mei Wang is Associate Professor at the University of Alabama at Birmingham, U.S. A. Eler 
research interests include: pre-service and in-service teachers’ IT training, computer networks in 
promoting student higher-order thinking skills, and pedagogical issues in instructional design. 


Dr. Der-Thanq Chen is Associate Professor with the Learning Sciences and Technologies Academic 
Group, National Institute of Education, Nanyang Technological University, Singapore. He is also the 
Teacher Education Research Track Leader within the Learning Sciences Lab. His research interests 
include: teacher education, community of practice, virtual learning environments and e-leaming. 


Journal of Asynchronous Learning Networks, Volume 12: Issue 3-4 


111 


